Current 3D content is based on a film source, which has a rate of 24 frame per second. Television broadcasts are at either a 50 Hz or a 60 Hz refresh rate. To convert a film source for television broadcast, there needs to be a frame rate conversion. This is typically accomplished by a frame rate conversion (FRC) process.
One way of performing FRC on a film source to a 60 Hz refresh rate is to use a 2:3 pulldown, in which a first frame is displayed twice and a second frame is displayed three times. This is repeated for each pair of frames in the film source. Because the frame rate for television is not the same as that for film, there is a motion judder problem (i.e., the viewer may notice that motion within the image is not smooth) introduced during the 2:3 pulldown.
Occlusion Problem
The output of the FRC process is a new (middle) frame that is placed between successive source frames. A problem with the new frame can arise if there is missing information regarding an obscured or hidden object in the background. If there is missing information, FRC cannot accurately produce the middle frame. This is referred to as the occlusion problem. If the middle frame is produced by motion estimation (determining a motion vector) and motion compensation (applying the motion vector to reduce the effects of motion), i.e., by the FRC process, then some immaterial images may be produced. This problem exists in both 2D images and in 3D images, where the hidden material can be in the left image, the right image, or both.
An example of an occlusion region is shown in FIG. 1, which shows five frames—a current frame (N), two previous frames (N−2, N−1), and two later frames (N+1, N+2). In normal FRC, an occlusion region shows no good matching motion vectors. If there is a motion vector field discontinuity in a homogeneous region, this could lead to potential occlusion regions because the correct motion vector could not be estimated due to the discontinuity. The motion vector discontinuity due to occlusion could be detected be comparing the motion vectors between frames N−2 and N−1 and between frames N−1 and N+1. To be able to handle the occlusion region correctly, more than four consecutive frames of data will be needed, creating a complex solution.
There is therefore a need to reduce judder when converting a film source, in particular with a 3D film source. There is also a need to address the occlusion problem in 3D sources.